Scan line interpolation device, image processing device, image display device, and scan line interpolation method

ABSTRACT

In the conversion of a video signal from interlaced to progressive scanning, the value of each pixel on an interpolated scan line is calculated by a procedure that includes calculating similarity values for pairs of pixel blocks located in point-symmetrical positions on opposite sides of the interpolated pixel; deciding whether similar edges are present in corresponding positions in the two pixel blocks constituting each pixel block pair; selecting an interpolation direction corresponding to the most similar pixel block pair among the pixel block pairs in which the similar edges are present; and using the pixels disposed at or closest to the centers of the two pixel blocks in this pixel block pair as reference pixels. Restricting diagonal interpolation directions to pixel block pairs in which similar edges are present improves the accuracy of the interpolation direction.

BACKGROUND OF THE INVENTION

1. Field of the Invention

This invention relates to the interpolation of scan lines to convert theresolution of an image, and in particular to the interpolation of scanlines during the conversion of a video signal from interlaced scanningto progressive scanning.

2. Description of the Related Art

In the interlaced scanning system, one frame of a video signal isdivided into two fields. When the frame is displayed, first one field isscanned; then the other field is scanned. The scan lines of the twofields occupy alternate positions on the display screen, so that atypical scan line of the first field is positioned adjacently betweentwo scan lines of the second field, and a typical scan line of thesecond field is positioned adjacently between two scan lines of thefirst field. In the progressive scanning system, a frame comprises onlyone field, and each scan line in the field is adjacent to other scanlines in the same field.

The conversion of a video signal from interlaced scanning to progressivescanning requires the separate generation of two video signal framesfrom the first and second fields of an interlaced video signal. Scanningconversion is therefore performed by interpolating scan lines betweenthe scan lines of each field of the interlaced video signal.

If the interlaced video signal is of a nonmoving image, so-calledinter-field interpolation is generally performed: the scan lines in onefield are inserted between the scan lines of the other field, so thatthe scan lines of both fields are used in each frame. If the interlacedvideo signal is of a moving image, so-called intra-field interpolationis performed: each frame is generated from the scan lines of just onefield, the interpolated scan lines being generated by averaging thevalues of picture elements (pixels) on the adjacent scan lines. Examplesare shown in FIG. 4 of Japanese Unexamined Patent ApplicationPublication (hereinafter, JP) H3-179890 and FIG. 1 of JP 2002-1122003.

An image may also be divided into a nonmoving part and a moving part,and inter-field interpolation and intra-field interpolation may beperformed to interpolate scan lines in the nonmoving part and movingpart, respectively, as disclosed in JP H3-179890.

One scanning conversion system, disclosed in JP H3-179890, interpolatesscan lines by generating pixels through vertical averaging of the pixelvalues on adjacent scan lines. If the image being converted includesdiagonal lines or edges oriented at an angle to the scan line direction,however, pixel interpolation by vertical pixel averaging alone producesblurred or jagged boundaries.

Another scanning conversion system, disclosed in JP 2002-1122003,eliminates blurred or jagged boundary lines by selecting the moststrongly correlated pair of pixels from among the pairs of pixelsdisposed at point-symmetrical positions on opposite sides of the pixelto be interpolated. Such a pair of pixels will be referred hereinafterto as a ‘pixel pair’.

The conventional scanning conversion systems described above both havedifficulties with images including fine lines oriented at small anglesto the scan line direction. That is, it is difficult to generate thepixels for these fine lines correctly on an interpolated scan line.

In the scanning conversion system disclosed in JP H3-179890, in whichinterpolated scan lines are generated by averaging the video signals(pixel values) of vertically adjacent scan lines, an interpolated pixelon an interpolated scan line is given the average value of the pixelimmediately above it and the pixel immediately below it. If theinterpolated pixel lies on a very fine line, however, and the pixelsimmediately above and below the interpolated pixel do not form part ofthe very fine line, then the fine line will disappear at this point.After scanning conversion, fine lines oriented at small angles to thescan lines tend to be displayed as discontinuous lines with a dotted ordashed appearance.

In the scanning conversion system disclosed in JP 2002-1122003, in whichcorrelations between pixel pairs are obtained, the level of correlationis determined from differences between the value of a pixel blockincluding a predetermined number of pixels surrounding one pixel in thepixel pair and the value of a pixel block including the same number ofpixels surrounding the other pixel in the pixel pair. The value of apixel block (the pixel block value) is calculated as the sum of thepixel values in the block, or as a weighted sum of these pixel values.With this method, however, the direction of very fine diagonal lines maybe unidentifiable because of identical differences between pixel blockvalues in two or more pixel pairs, in which case the best pixel pair touse for generating the interpolated pixel cannot be selectedunambiguously. After scanning conversion, the fine line may therefore bedisplayed as a discontinuous line.

SUMMARY OF THE INVENTION

An object of the present invention is to provide a scan lineinterpolation device and method by which scanning conversion can beperformed without generating blurred or jagged boundaries at diagonallines or edges and without generating discontinuities in fine lines thatmake a small angle with the scan lines.

The invented scan line interpolation device interpolates a scan linebetween two adjacent scan lines by using pixels on the two adjacent scanlines and at least two further scan lines adjacent to those scan lines.Each interpolated pixel is generated with reference to pixels selectedfrom these scan lines by a pattern matching means, a similar edgedecision means, and an interpolation direction decision means.

The pattern matching means calculates pattern similarity values for aplurality of pixel block pairs. The similarity value of each pixel blockpair is calculated from the values of the pixels in its two constituentpixel blocks by, for example, summing the absolute differences betweenpixels in corresponding positions in the two pixel blocks.

The similar edge decision means decides whether similar edges arepresent in corresponding positions in the two pixel blocks constitutingeach pixel block pair. The decision is made on the basis of differencesbetween the values of pairs of mutually adjacent pixels alignedperpendicular to the scan lines in the two pixel blocks.

From among the pixel block pairs in which similar edges are present, theinterpolation direction decision means selects the pixel block pairhaving the pattern similarity value indicating the greatest similarity,thereby selecting an interpolation direction.

An interpolation means generates the interpolated pixel with referenceto the pixels disposed closest to the centers of the two pixel blocks inthe pixel block pair selected by the interpolation direction decisionmeans.

BRIEF DESCRIPTION OF THE DRAWINGS

In the attached drawings:

FIG. 1 is a block diagram illustrating a scan line interpolation deviceaccording to a first embodiment of the invention;

FIG. 2 schematically illustrates an arrangement of pixels in a videosignal in the first embodiment;

FIG. 3 schematically illustrates another arrangement of pixels in avideo signal in the first embodiment;

FIG. 4 illustrates pixel blocks that can be used in generating aninterpolated pixel;

FIG. 5 illustrates other pixel blocks that can be used in generating aninterpolated pixel;

FIG. 6 illustrates further pixel blocks that can be used in generatingan interpolated pixel;

FIG. 7 illustrates still further pixel blocks that can be used ingenerating an interpolated pixel;

FIG. 8 is a block diagram illustrating a scan line interpolation deviceaccording to a third embodiment of the invention;

FIG. 9 is a block diagram illustrating a scan line interpolation deviceaccording to a fourth embodiment of the invention;

FIG. 10 is a block diagram illustrating a scan line interpolation deviceaccording to a fifth embodiment of the invention;

FIG. 11 is a block diagram illustrating an image processing deviceaccording to a sixth embodiment of the invention; and

FIG. 12 is a block diagram illustrating an image display deviceaccording to a seventh embodiment of the invention.

DETAILED DESCRIPTION OF THE INVENTION

Embodiments of the invention will now be described with reference to theattached drawings, in which like elements are indicated by likereference characters. The terminology introduced above will be used tosimplify the descriptions: the term ‘pixel block’ will mean a regionconsisting of a predetermined number of pixels, and the term ‘pixelblock pair’ will mean two pixel blocks disposed at point-symmetricalpositions on opposite sides of the interpolated pixel.

FIRST EMBODIMENT

Referring to FIG. 1, a scan line interpolation device according to afirst embodiment of the invention includes first, second, and thirddelay circuits 1, 2, 3, a pattern matching circuit 4, a similar edgedecision circuit 5, a diagonal line recognition circuit 6, aninterpolation circuit 7, and an input terminal 200. The diagonal linerecognition circuit 6 functions as an interpolation direction decisionmeans.

An interlaced video signal input from the input terminal 200 is suppliedto the first delay circuit 1, in which it is delayed by one horizontalscan period (1H), and to the pattern matching circuit 4 and the similaredge decision circuit 5. The video signal delayed by the first delaycircuit 1 is supplied to the second delay circuit 2, in which it isfurther delayed by 1H, and to the pattern matching circuit 4, similaredge decision circuit 5, and interpolation circuit 7. The video signaldelayed by the second delay circuit 2 is supplied to the third delaycircuit 3, in which it is further delayed by 1H, and to the patternmatching circuit 4, similar edge decision circuit 5, and interpolationcircuit 7. The video signal delayed by the third delay circuit 3 issupplied to the pattern matching circuit 4 and similar edge decisioncircuit 5.

FIG. 2 schematically illustrates an arrangement of pixels on fourconsecutive scan lines. Scan line A is formed by the video signal outputfrom the third delay circuit 3; scan line B is formed by the videosignal output from the second delay circuit 2; scan line C is formed bythe video signal output from the first delay circuit 1; scan line D isformed by the video signal input from the input terminal 200.

The interpolation of scan line P between scan lines B and C will now bedescribed. The value of interpolated pixel P0 on interpolated scan lineP will be denoted P(0). The value of the pixel B0 immediately aboveinterpolated pixel P0 on scan line B will be denoted B(0). The value ofthe pixel A0 immediately above pixel B0 on scan line A will be denotedA(0). The value of the pixel C0 immediately below interpolated pixel P0on scan line C will be denoted C(0). The value of the pixel DOimmediately below pixel C0 on scan line D will be denoted D(0).

The n-th pixel from pixel A0 in the scanning direction (to the right inthe drawings) on scan line A will be denoted An, and its pixel valuewill be denoted A(n), where n is an arbitrary positive integer. The n-thpixel from pixel A0 in the opposite direction (to the left in thedrawings) will be denoted A-n, and its pixel value A(-n). Accordingly,the pixels on scan line A have values A(0), A(1), A(2), A(3) and so onin the scanning direction, and A(-1), A(-2), A(-3) and so on in theopposite direction. The pixels on scan line B have values B(0), B(1),B(2), B(3) and so on in the scanning direction, and B(-1), B(-2), B-3)and so on in the opposite direction. The pixels on scan line C havevalues C(0), C(1), C(2), C(3) and so on in the scanning direction, andC(-1), C(-2), C(-3) and so on in the opposite direction. The pixels onscan line D have values D(0), D(1), D(2), D(3) and so on in the scanningdirection, and D(-1), D(-2), D(-3) and so on in the opposite direction.

The direction of the straight line connecting pixels Bn and C-n inpoint-symmetrical positions on opposite sides of interpolated pixel P0will be referred to as direction n. For instance, the line through P0connecting B0 and C0 has direction 0; the line through P0 connecting B1and C-1 has direction 1; the line through P0 connecting B2 and C-2 hasdirection 2; the line through P0 connecting B-1 and C1 has direction −1;the line through P0 connecting B-2 and C2 has direction −2. FIG. 2 usesarrows to indicate the integer-valued directions from −4 to 4.

The direction of the straight line connecting the midpoint between Bnand Bn+1 and the midpoint between C−n and C−(n+1) in point-symmetricalpositions on opposite sides of interpolated pixel P0 will be referred toas direction n.5. For instance, the line through P0 connecting themidpoint between B0 and B1 and the midpoint between C0 and C-1 hasdirection 0.5; the line through P0 connecting the midpoint between B1and B2 and the midpoint between C-1 and C-2 has direction 1.5; the linethrough P0 connecting the midpoint between B0 and B-1 and the midpointbetween C0 and C1 has direction −0.5; the line through P0 connecting themidpoint between B-1 and B-2 and the midpoint between C1 and C2 hasdirection −1.5. FIG. 3 uses arrows to indicate the half-integer-valueddirections from −3.5 to 3.5.

The function of the pattern matching circuit 4 will be described next.The pattern matching circuit 4 calculates the similarity values for aplurality of pixel block pairs disposed in point-symmetrical positionsin directions −4 to 4 on opposite sides of the interpolated pixel P0.For an integer-valued direction n, the similarity value is obtained froma nine-pixel block centered on Bn and a nine-pixel block centered onC−n.

As an example, the calculation of a similarity value S(3) for the pixelblock pair in direction 3 in FIG. 4 will be described. This pixel blockpair comprises the two pixel blocks 401 and 402 enclosed in dash-dottedboxes.

Pixel block 401 includes nine pixels A2, A3, A4, B2, B3, B4, C2, C3, C4,with B3 disposed at the center. Pixel block 402 includes nine pixelsB-4, B-3, B-2, C-4, C-3, C-2, D-4, D-3, D-2, with C-3 disposed at thecenter.

First, the absolute value of the difference between each pixel in pixelblock 401 and the corresponding pixel in pixel block 402 is calculated,as follows.d0=|A(2)−B(-4)|  (1)d1=|A(3)−B(-3)|  (2)d2=|A(4)−B(-2)|  (3)d3=|B(2)−C(-4)|  (4)d4=|B(3)−C(-3)|  (5)d5=|B(4)−C(-2)|  (6)d6=|C(2)−D(-4)|  (7)d7=|C(3)−D(-3)|  (8)d8=|C(4)−D(-2)|  (9)

The similarity value S(3) for direction 3 is the total of these absolutedifferences d0 to d8.S(3)=d0+d1+d2+d3+d4+d5+d6+d7+d8  (10)

It can be seen from equations (1) to (10) that the similarity value S(3)approaches zero as the similarity between pixel blocks 401 and 402increases, and that the similarity value S(3) increases as thesimilarity between the two pixel blocks decreases.

The pattern matching circuit 4 calculates similarity values for thedirections n in the range from −4 to 4, and supplies all the calculatedsimilarity values S(n) to the diagonal line recognition circuit 6.

The similarity value S(3) may also be obtained as a weighted sum of d0to d8, a higher weight being assigned to the difference between thecentral pixels than to the differences between the surrounding pixels,as shown below.S(3)=0.1×d0+0.1×d1+0.1×d2+0.1×d3+0.2×d4+0.1×d5+0.1×d6+0.1×d7+0.1×d8  (11)

The pixel blocks used to calculate a similarity value need not always berectangular blocks three pixels wide by three pixels high as shown inFIG. 4. The pixel blocks may be rectangular blocks two pixels wide bythree pixels high, as shown in FIGS. 5 and 7, or rhombic blocks, asshown in FIG. 6.

The calculation of similarity values need not be carried out asindicated above. Any calculation method that numerically expresses thesimilarity of the patterns of pixel values in the two pixel blocks maybe used.

The pixel blocks constituting different pixel block pairs may includedifferent numbers of pixels. In FIG. 5, for example, the nine-pixelblocks 401, 402 centered on B3 and C-3 may be used to calculate asimilarity value for direction 3, and the six-pixel blocks 401(b),402(b) centered on the midpoint between B0 and B1 and the midpointbetween C0 and C-1 may be used to calculate a similarity value fordirection 0.5. The absolute value of the difference between each pixelin pixel block 401(b) and the corresponding pixel in pixel block 402(b)is calculated as follows.d0(b)=|A(0)−B(-1)|  (1-b)d1(b)=|A(1)−B(0)|  (2-b)d2(b)=|B(0)−C(-1)|  (3-b)d3(b)=|B(1)−C(0)|  (4-b)d4(b)=|C(0)−D(-1)|  (5-b)d5(b)=|C(1)−D(0)|  (6-b)

If the total of d0 to d8 in equations (1) to (9) is used as thesimilarity value S(3) for direction 3 and the total of d0(b) to d5(b) inequations (1-b) to (6-b) is used as the similarity value S(0.5) fordirection 0.5, then similarity value S(3) is the sum of ninedifferences, while similarity value S(0.5) is the sum of only sixdifferences. Therefore, before the two similarity values are compared,they are multiplied by coefficients determined by the number of pixelsper block, such as 1/9 for a nine-pixel block and 1/6 for a six-pixelblock.S(3)=(d0+d1+d2+d3+d4+d5+d6+d7+d8)×1/9  (12)S(0.5)=(d0(b)+d1(b)+d2(b)+d3(b)+d4(b)+d5(b))×1/6  (13)

These weighted similarity values S(3) and S(0.5) also approach zero asthe similarity between the pixel blocks being compared increases, andincrease as the similarity between the pixel blocks decreases. Thepattern matching circuit 4 calculates similarity values S(n) in this wayfor all integer-valued and half-integer-valued directions n from −4 to4, and supplies all the calculated similarity values to the diagonalline recognition circuit 6.

The function of the similar edge decision circuit 5 will next bedescribed. The similar edge decision circuit 5 decides whether similaredges are present in the two pixel blocks constituting each pixel blockpair. The operation of the similar edge decision circuit 5 for decidingthe presence or absence of similar edges will be described withreference to pixel blocks 401 and 402 in FIG. 4.

The similar edge decision circuit 5 calculates differences between thevalues of pairs of adjacent pixels at, above, and below the centers ofthese pixel blocks. Each pair of pixels is vertically aligned with thecenter of the block, thus being aligned perpendicular to the scan lines.For pixel block 401, the following two differences are calculated.v1=A(3)−B(3)  (14)v2=B(3)−C(3)  (15)

For the other pixel block 402, the following two differences arecalculated.v3=B(-3)−C(-3)  (16)v4=C(-3)−D(-3)  (17)

Then the similar edge decision circuit 5 checks whether the followingconditions 1-1 and 1-2 are satisfied, where Th is a predeterminedthreshold:

Condition 1-1: |v1| is greater than Th; |v3| is greater than Th; and v1and v3 have the same sign.

Condition 1-2: |v2| is greater than Th; |v4| is greater than Th; and v2and v4 have the same sign.

If either condition 1-1 or condition 1-2 is satisfied, the similar edgedecision circuit 5 decides that similar edges are present in pixelblocks 401 and 402, sets a signal V(3) to ‘1’ to indicate the decisionresult, and supplies this signal value to the diagonal line recognitioncircuit 6. If neither condition 1-1 nor condition 1-2 is satisfied, thesimilar edge decision circuit 5 decides that similar edges are notpresent in pixel blocks 401 and 402, sets signal V(3) to ‘0’, andsupplies this signal value to the diagonal line recognition circuit 6.

For each integer n from −4 to 4, the similar edge decision circuit 5uses conditions 1-1 and 1-2 to decide whether similar edges are presentin the two pixel blocks constituting the pixel block pair aligned indirection n, sets a signal V(n) to ‘0’ or ‘1’ to indicate the result ofthe decision, and supplies the signal value V(n) to the diagonal linerecognition circuit 6.

The same decision procedure can be used for the five-pixel blocks shownin FIG. 6. A slightly modified procedure can be used for pixel blockpairs aligned in half-integer directions, such as the six-pixel blocksin FIG. 7.

Further modifications of the similar edge decision procedure are alsopossible. For example, the decision can be based on variations in thevalues of vertically aligned triplets of pixels instead of variations inthe values of vertically aligned pairs of pixels.

The operation of the diagonal line recognition circuit 6 will now bedescribed. The diagonal line recognition circuit 6 receives thesimilarity values S(n) from the pattern matching circuit 4 and similaredge decision signals V(n) from the similar edge decision circuit 5 forall directions n from −4 to 4. If any similar edge decision signal V(n)is set to ‘1’, indicating that similar edges are present, then fromamong all directions n for which V(n) is set to ‘1’, the diagonal linerecognition circuit 6 selects the direction n having the similarityvalue S(n) nearest zero (denoting the greatest similarity), and sendsthis direction value n to the interpolation circuit 7 as a directionsignal (dir), indicating that there is a diagonal line or edge orientedin direction n. If all of the similar edge decision signals V(n) are‘0’, indicating that no similar edges are present, the direction signalis set to zero (dir=0) to indicate that there is no diagonal line oredge.

Next, the operation of the interpolation circuit 7 will be described.The interpolation circuit 7 uses the direction signal (dir) receivedfrom the diagonal line recognition circuit 6 to select the pixels onscan lines B and C from which to generate the interpolated pixel P0. Ifthe value of the direction signal is an integer n, the value P(0) of theinterpolated pixel P0 is normally calculated from pixel values B(n) andC(−n) by the following formula:P(0)={B(n)+C(−n)}/2  (19)

If the value of the direction signal is three (dir=3), for example, thevalue P(0) of pixel P0 is calculated from the values B(3) and C(-3) ofpixels B3 and C-3.P(0)={B(3)+C(-3)}/2  (18)

If there are no pixels at the centers of the pixel blocks correspondingto direction n (for example, if n is not an integer), the pixels on scanlines B and C closest to the centers of the blocks are used instead. Forthe example shown in FIG. 7, the value P(0) of interpolated pixel P0 iscalculated from the mean value of pixels B2 and B3 on scan line B andthe mean value of pixels C-2 and C-3 on scan line C.P(0)={(B(2)+B(3))/2+(C(-2)+C(-3))/2}/2  (20)

More generally, if the value of the direction signal (dir) is n.5, thevalue P(0) of interpolated pixel P0 is calculated as follows:P(0)={(B(n)+B(n+1))/2+(C(−n)+C(−n−1))/2}/2  (21)

The scan line interpolation device of the first embodiment canaccurately determine the direction of a line even if the line has anarrow width and is oriented at a small angle to the scan lines, becausethe determination is made on the basis of both pattern similarity andedge similarity of the two pixel blocks constituting a pixel block pair.In comparison with the conventional conversion system in which thedirections of diagonal lines and edges are determined from pixel blockvalues, the scan line interpolation device of the first embodiment cansignificantly reduce blurred or jagged edges and reduce discontinuitiesin fine lines.

The scan line interpolation device of the first embodiment has beendescribed as hardware, but needless to say, it can be implemented insoftware.

SECOND EMBODIMENT

The second embodiment is identical to the first embodiment, except thatin order to improve direction identification accuracy, the diagonal linerecognition circuit 6 is modified to select the direction dir1 havingthe smallest similarity value S(n) (the greatest similarity) and thedirection dir2 having the second smallest similarity value S(n) (thesecond greatest similarity), from among the directions for which thesimilar edge decision signal V(n) is set to ‘1’. If the absolute valueof the difference between the similarity values of dir1 and dir2 doesnot exceed a predetermined amount, the diagonal line recognition circuit6 identifies dir1 as the diagonal line direction and outputs thedirection signal accordingly (dir=dir1). Otherwise, the diagonal linerecognition circuit 6 decides that there is no diagonal line or edge,and sends the interpolation circuit 7 a zero direction signal (dir=0).

THIRD EMBODIMENT

In the scan line interpolation device of the first or second embodiment,the diagonal line recognition circuit may occasionally recognize adiagonal line or edge to which the interpolated pixel does not belongbecause the interpolated pixel forms part of a small intervening object.In this case the interpolated pixel should be generated from otherpixels in the intervening object, more specifically, from the pixelsimmediately above and below the interpolated pixel. To deal with thiscase, the third embodiment adds an exception decision circuit to theconfiguration of the first or second embodiment.

FIG. 8 is a block diagram illustrating the scan line interpolationdevice in the third embodiment. The delay circuits 1, 2, 3, patternmatching circuit 4, similar edge decision circuit 5, and interpolationcircuit 7 are the same as in the first embodiment and will not bedescribed below. The diagonal line recognition circuit 6 is modified toaccept input from the newly added exception decision circuit 8.

The exception decision circuit 8 receives the video signal input fromthe input terminal 200 and the video signals output from the first,second, and third delay circuits 1, 2, 3, and performs subtractionoperations to obtain the following quantities:hl1=A(-1)−A(0)  (22)hl2=B(-1)−B(0)  (23)hl3=C(-1)−C(0)  (24)hl4=D(-1)−D(0)  (25)hr1=A(1)−A(0)  (26)hr2=B(1)−B(0)  (27)hr3=C(1)−C(0)  (28)hr4=D(1)−D(0)  (29)

The exception decision circuit 8 then checks whether the followingconditions 2-1 to 2-6, 3-1, and 3-2 are satisfied, where Th2 and Th3 arepredetermined thresholds:

Condition 2-1: |hl1| is greater than Th2; |hl2| is greater than Th2; andhl1 and hl2 have the same sign.

Condition 2-2: |hl2| is greater than Th2; |hl3| is greater than Th2; andhl2 and hl3 have the same sign.

Condition 2-3: |hl3| is greater than Th2; |hl4| is greater than Th2; andhl3 and hl4 have the same sign.

Condition 2-4: |hr1| is greater than Th2; |hr2| is greater than Th2; andhr1 and hr2 have the same sign.

Condition 2-5: |hr2| is greater than Th2; |hr3| is greater than Th2; andhr2 and hr3 have the same sign.

Condition 2-6: |hr3| is greater than Th2; |hr451 is greater than Th2;and hr3 and hr4 have the same sign.

Condition 3-1: |hl2| is greater than Th3; |hl3| is greater than Th3; andhl2 and hl3 have opposite signs.

Condition 3-2: |hr2| is greater than Th3; |hr3| is greater than Th3; andhr2 and hr3 have opposite signs.

Conditions 2-1 to 2-6 indicate the presence of a vertical edge at one orboth of the pixels vertically adjacent to the interpolated pixel P0.Given the presence of such a vertical edge, if a diagonal line or edgeis also recognized, the interpolated pixel P0 is likely to belong to asmall object interrupting the diagonal line or edge. Conditions 3-1 and3-2 also indicate that any recognized diagonal line or edge is likely tobe interrupted in the immediate vicinity of the interpolated pixel.

Accordingly, if at least one of conditions 2-1 to 2-6, 3-1, and 3-2 issatisfied, the exception decision circuit 8 sets an exception decisionsignal EX to ‘1’ to instruct the diagonal line recognition circuit 6 todisregard diagonal lines and edges. If none of conditions 2-1 to 2-6,3-1, and 3-2 is satisfied, the exception decision circuit 8 sets theexception decision signal EX set to ‘0’. The exception decision signalEX is sent to the diagonal line recognition circuit 6.

If the exception decision signal EX is ‘1’, the diagonal linerecognition circuit 6 sets the direction signal to zero (dir=0)regardless of the similarity values S(n) supplied from the patternmatching circuit 4 and the similar edge decision signals V(n) suppliedfrom the similar edge decision circuit 5. The zero direction signalinforms the interpolation circuit 7 that no diagonal line or edge ispresent.

If the exception decision signal EX is ‘0’, the diagonal linerecognition circuit 6 selects an interpolation direction on the basis ofthe similarity values S(n) supplied from the pattern matching circuit 4and the similar edge decision signals V(n) supplied from the similaredge decision circuit 5 as in the first or second embodiment, sets thedirection signal (dir) accordingly, and outputs the direction signal tothe interpolation circuit 7.

The operation of the exception decision circuit described above may bemodified. For example, the exception decision circuit may examine onlythe pixels on the two scan lines adjacent to the interpolated pixel,calculate only hl2, hr2, hl3, and hr3, and test only conditions 2-2,2-5, 3-1, and 3-2, or modified forms thereof.

The exception decision circuit in the scan line interpolation device ofthe third embodiment enables pixels in small objects to be interpolatedcorrectly even when the small object interrupts a diagonal line or edge.

FOURTH EMBODIMENT

In the first, second, and third embodiments, the interpolation circuitaccepts the interpolation direction selected by the diagonal linerecognition circuit, and interpolates a pixel value from the values ofnearby pixels aligned with the interpolated pixel in this direction. Toreduce the chance that an incorrectly selected interpolation directionmay be used, the fourth embodiment adds a circuit that recognizes andeliminates anomalous interpolation directions.

FIG. 9 is a block diagram illustrating the scan line interpolationdevice in the fourth embodiment. The delay circuits 1, 2, 3, patternmatching circuit 4, similar edge decision circuit 5, diagonal linerecognition circuit 6, and interpolation circuit 7 are the same as inthe first embodiment and will not be described in detail. An isolateddirection correction circuit 9 is inserted between the diagonal linerecognition circuit 6 and interpolation circuit 7 to set theinterpolation direction to zero unless it is close to the interpolationdirection of an adjacent interpolated pixel on the same interpolatedscan line.

The diagonal line recognition circuit 6 receives similarity values S(n)from the pattern matching circuit 4 and similar edge decision signalsV(n) from the similar edge decision circuit 5, and sets the directionsignal (dir) to a value indicating the diagonal line directionaccordingly. The isolated direction correction circuit 9 receives thedirection signal from the diagonal line recognition circuit 6 andsupplies a corrected direction signal (cdir) to the interpolationcircuit 7.

The operation of the isolated direction correction circuit 9 will now bedescribed. The isolated direction correction circuit 9 compares thedirection signals dir(-1), dir(0), and dir(1) received from the diagonalline recognition circuit 6 for pixels P-1, P0, and P1 on an interpolatedscan line by calculating the following absolute differences:dL=|dir(-1)−dir(0)|  (30)dR=|dir(1)−dir(0)|  (31)

Then the isolated direction correction circuit 9 decides whether thefollowing conditions 4-1 and 4-2 are satisfied, in which Th1 is apredetermined threshold:

Condition 4-1: dL≦Th1

Condition 4-2: dR≦Th1

If either of these two conditions is satisfied, the isolated directioncorrection circuit 9 sends the interpolation circuit 7 the directionsignal dir(0) provided by the diagonal line recognition circuit 6 forinterpolated pixel P0 as the corrected diagonal line direction signal(cdir). If neither of the two conditions is satisfied, the isolateddirection correction circuit 9 sends the interpolation circuit 7 acorrected diagonal line direction signal set to zero (cdir=0). Theinterpolation circuit 7 generates the value P(0) of interpolated pixelP0 in the same was as in the first embodiment, using the correcteddirection signal (cdir) instead of the signal (dir) output by thediagonal line recognition circuit 6.

Accordingly, if the interpolation direction selected for an interpolatedpixel differs greatly from the interpolation directions of both adjacentinterpolated pixels on the same interpolated scan line, theinterpolation direction of the interpolated pixel is altered to zero.This prevents the scan line interpolation device of the fourthembodiment from recognizing spurious diagonal lines resulting from noiseor the like.

FIFTH EMBODIMENT

The scan line interpolation devices in the first four embodimentsgenerate interpolated pixels from other pixels in the same field. Thescan line interpolation device of the fifth embodiment decides whethereach interpolated pixel is in a nonmoving part or moving part of theimage and performs different types of interpolation accordingly. If theinterpolated pixel is in a moving part, intra-field interpolation isused generate the interpolated pixel value as described in the firstembodiment. If the interpolated pixel is in a nonmoving part,inter-field interpolation is performed by using the corresponding pixelvalue from the preceding field.

Referring to FIG. 10, the scan line interpolation device according tothe fifth embodiment comprises an input terminal 200, a still-imageinterpolator 201, a moving-image interpolator 202, a motion detector203, a combiner 204, and a time-axis converter 205.

An interlaced video signal input from the input terminal 200 is suppliedto the still-image interpolator 201, moving-image interpolator 202,motion detector 203, and time-axis converter 205. The still-imageinterpolator 201 has a field memory and generates interpolated pixelvalues by performing inter-field interpolation, e.g., by interpolatingthe scan lines of each field between the scan lines of the next field.The moving-image interpolator 202 generates interpolated pixel values byperforming intra-field interpolation as described in the firstembodiment. The motion detector 203 uses a frame memory to obtain thedifference between the input video signal and the video signal for thepreceding frame, both fields of which are stored in the frame memory.Motion is detected when the difference is larger than a predeterminedthreshold, and a motion detection signal is output accordingly.

The interpolated pixel values generated by the still-image interpolator201 and the moving-image interpolator 202 and the motion detectionsignal output from the motion detector 203 are supplied to the combiner204. The combiner 204 combines the two interpolated signals into asingle interpolated signal according to the motion detection signal. Forparts of the image in which the motion detection signal indicates thatmotion is present, the combiner 204 outputs the interpolated pixelvalues generated by the moving-image interpolator 202. For other parts,the combiner 204 outputs the interpolated pixel values generated by thestill-image interpolator 201.

The time-axis converter 205 aligns the time axes of the input videosignal and the output of the combiner 204, inserts the combinedinterpolated scan lines generated by the combiner 204 between the scanlines of the input video signal, and outputs a progressively scannedvideo signal.

The scan line interpolation device of the fifth embodiment generatesinterpolated pixels by intra-field interpolation in moving parts of animage and by inter-field interpolation in nonmoving parts, so that ahigh-quality image can be displayed over the whole screen.

The moving-image interpolator 202 may be modified to operate as in thesecond, third, or fourth embodiment instead of the first embodiment.

SIXTH EMBODIMENT

The sixth embodiment of the invention is an image processing devicehaving an image adjustment function and a scan line interpolationfunction. Referring to FIG. 11, the image processing device 11 includesa scan line interpolator 14, an image processor 15, an input terminal18, and an output terminal 19. The scan line interpolation device of anyof first to fifth embodiments may be used as the scan line interpolator14. The operation of the image processor 15 will be described next.

When an interlaced video signal output from a video signal source suchas a digital versatile disc (DVD) player or video cassette recorder(VCR) is supplied from the input terminal 18, the image processor 15performs image adjustments (for example, edge enhancement, gammaadjustment, and adjustments of screen size, contrast, brightness, color,etc.), and sends the scan line interpolator 14 the adjusted interlacedvideo signal.

The scan line interpolator 14 interpolates scan lines as described inthe preceding embodiments, and supplies the interpolated scan lines tothe image processor 15. The image processor 15 inserts the interpolatedscan lines into the adjusted interlaced video signal and outputs theresulting progressively scanned signal from the output terminal 19 to anexternal image display device (not shown).

The image processing device 11 may also include a video reproductiondevice for reproducing a video signal recorded on a DVD, magnetic tape,hard disk, or the like, and this image reproduction device may supplythe interlaced video signal to the input terminal 18.

The image processing device of the sixth embodiment can perform imageadjustments and scanning conversion without generating blurred or jaggedboundaries and without generating discontinuities in fine lines, evenfor an image including diagonal lines or edges, or fine lines that makea small angle with the scan lines.

SEVENTH EMBODIMENT

The seventh embodiment is an image display device that includes an imageprocessing device with a scan line interpolation function and an imageadjustment function. As shown in FIG. 12, the image display device 10includes an image processing device 11, a receiving antenna 12, a tuner13, a driver 16, and a cathode ray tube (CRT) 17. The image processingdevice 11 includes an image processor 15 and a scan line interpolator 14as described in the sixth embodiment. The image display device 10operates as follows.

The receiving antenna 12 receives a television broadcast signal andsupplies it to the tuner 13. The tuner 13 performs tuning,intermediate-frequency amplification, and detection, and supplies anNTSC video signal to the image processing device 11. (NTSC is aninterlaced video standard developed by the National Television SystemsCommittee.) In the image processing device 11, the image processor 15performs image adjustments (such as edge enhancement, gamma adjustment,and adjustments of screen size, contrast, brightness, color, etc.), andsends the adjusted interlaced video signal to the scan line interpolator14, as in the sixth embodiment. The scan line interpolator 14interpolates scan lines as in the sixth embodiment, and supplies theinterpolated scan lines to the image processor 15. The image processor15 then outputs a progressively scanned signal to the driver 16, whichdrives the CRT 17 to display the image.

The CRT 17 may be replaced by any other type of display device, such asa liquid crystal display (LCD), electroluminescent (EL) display, plasmadisplay panel (PDP), or liquid crystal on silicon (LCOS) display. Thetuner 13 is not always necessary; the interlaced video signal may besupplied to the image processor 15 directly from a video signal sourcesuch as a DVD player or a VCR.

The image display device of the seventh embodiment can perform imageadjustments and scanning conversion and can display pictures withoutgenerating blurred or jagged boundaries at diagonal lines or edges andwithout generating discontinuities in fine lines, even if the imagereceived from the antenna includes, or fine lines forming a small anglewith the scan line direction.

The present invention is not limited to the preceding embodiments. A fewvariations have been mentioned above, and those skilled in the art willrecognize that further variations are possible within the scope definedby the appended claims.

1. A scan line interpolation device for generating an interpolated pixelon an interpolated scan line from pixels disposed on at least fourconsecutive scan lines, including at least two scan lines on each sideof the interpolated scan line, the scan line interpolation devicecomprising: a pattern matching means for calculating pattern similarityvalues for a plurality of pixel block pairs, each pixel block pairincluding two pixel blocks disposed in point-symmetrical positions onopposite sides of the interpolated pixel, each of the two pixel blocksincluding an identical number of said pixels, the pattern similarityvalue of each pixel block pair being calculated from values of thepixels in its two constituent pixel blocks; a similar edge decisionmeans for deciding whether similar edges are present in correspondingpositions in the two pixel blocks constituting each said pixel blockpair, on the basis of differences between the pixel values of pixelsdisposed on mutually adjacent scan lines in one of the pixel blocks,aligned in a direction perpendicular to the scan lines, and differencesbetween the values of pixels disposed in corresponding positions in theother pixel block in the same pixel block pair; an interpolationdirection decision means for selecting an interpolation directioncorresponding to the positions of the two pixel blocks in a pixel blockpair having a greatest similarity, as calculated by the pattern matchingmeans, among the pixel block pairs in which the similar edge decisionmeans decides that similar edges are present; and an interpolation meansfor generating the interpolated pixel by using, as reference pixels, thepixels disposed closest to the centers of the two pixel blocks in thepixel block pair corresponding to the interpolation direction selectedby the interpolation direction decision means.
 2. The scan lineinterpolation device of claim 1, wherein: if there are pixels at thecenters of the two pixel blocks in the pixel block pair corresponding tothe selected interpolation direction, the interpolation means generatesthe interpolated pixel by using the two pixels disposed at the centersas reference pixels; and if no pixels are disposed at the centers ofsaid two pixel blocks, the interpolation means generates theinterpolated pixel by using a plurality of pixels in each of said twopixel blocks as reference pixels.
 3. The scan line interpolation deviceof claim 2, wherein if no pixels are disposed at the centers of the twopixel blocks in the pixel block pair corresponding to the selectedinterpolation direction, the interpolation means uses, as referencepixels, two pixels closest to the center of each of the two pixelblocks.
 4. The scan line interpolation device of claim 1, wherein foreach pixel block pair, the pattern matching means calculates absolutevalues of difference between the values of pixels in mutuallycorresponding positions in the two pixel blocks constituting the pixelblock pair, takes a sum of the calculated absolute values, and outputsthe sum as the pattern similarity value of the pixel block pair.
 5. Thescan line interpolation device of claim 1, wherein for each pixel blockpair, the pattern matching means calculates absolute values ofdifference between the values of pixels in mutually correspondingpositions in the two pixel blocks constituting the pixel block pair,takes a sum of the calculated absolute values, weights the sum accordingto the number of pixels per block, and outputs the sum as the patternsimilarity value of the pixel block pair.
 6. The scan line interpolationdevice of claim 1, wherein for each pixel block pair, the patternmatching means calculates absolute values of difference between thevalues of pixels in mutually corresponding positions in the two pixelblocks constituting the pixel block pair, weights the absolute values,takes a sum of the weighted absolute values, and outputs the sum as thepattern similarity value of the pixel block pair.
 7. The scan lineinterpolation device of claim 1, wherein the similar edge decision meansdecides that similar edges are present when the following threeconditions are all satisfied: the difference between the values of twopixels disposed on mutually adjacent scan lines in one constituent pixelblock in a pixel block pair, aligned perpendicularly to the scan lines,is greater than a predetermined value; the difference between the pixelvalues of two pixels disposed in corresponding positions in the otherconstituent pixel block in the same pixel block pair is greater than thepredetermined value; and the two differences have the same sign.
 8. Thescan line interpolation device of claim 1, wherein the interpolationdirection selected by the interpolation direction decision means is thedirection of a line joining the centers of the two pixel blocks in thepixel block pair having the greatest similarity, among the pixel blockpairs in which the similar edge decision means decides that similaredges are present.
 9. The scan line interpolation device of claim 1,wherein the interpolation direction decision means selects aninterpolation direction perpendicular to the scan lines if the directionof a line joining the centers of the two pixel blocks in the pixel blockpair having the greatest similarity differs from the direction of a linejoining the centers of the two pixel blocks in the pixel block pairhaving a second greatest similarity, among the pixel block pairs inwhich the similar edge decision means decides that similar edges arepresent, by more than a predetermined amount.
 10. The scan lineinterpolation device of claim 1, further comprising an exceptiondecision means for testing for the presence of an edge perpendicular tothe scan lines by taking differences between at least the values of thetwo pixels adjacent to the interpolated pixel in a directionperpendicular to the scan lines and pixels adjacent to said two pixelsin the direction of the scan lines, wherein if the exception decisionmeans finds an edge perpendicular to the scan lines, the interpolationdirection decision means disregards the pattern similarities calculatedby the pattern matching means and the decisions made by the similar edgedecision means, and selects the direction perpendicular to the scanlines as the interpolation direction.
 11. The scan line interpolationdevice of claim 1, further comprising an isolated direction correctionmeans for changing the interpolation direction selected by theinterpolation direction decision means to a direction perpendicular tothe scan lines if said interpolation direction differs by more than apredetermined amount from the interpolation directions selected by theinterpolation direction decision means for the adjacent interpolatedpixels on both sides of the interpolated pixel on said interpolated scanline.
 12. The scan line interpolation device of claim 1, furthercomprising: a still-image interpolator for generating interpolatedpixels by performing inter-field interpolation; a motion detector fordeciding whether an interpolated pixel is included in a nonmoving imagepart or a moving image part; and a combiner for combining theinterpolated pixels generated by the still-image interpolator with theinterpolated pixels generated by the interpolation means according tothe decision made by the motion detector.
 13. An image processing devicecomprising the scan line interpolation device of claim 1, furthercomprising an image processor for making image adjustments to an inputvideo signal and supplying the adjusted input video signal to the scanline interpolation device.
 14. An image display device comprising theimage processing device of claim 13, further comprising a display meansfor displaying an image according to the video signal supplied from theimage processor and interpolated pixels generated by the scan lineinterpolation device in the image processing device.
 15. A scan lineinterpolation method for generating an interpolated pixel on aninterpolated scan line from pixels disposed on at least four consecutivescan lines, including at least two scan lines on each side of theinterpolated scan line, the method comprising: calculating patternsimilarity values for a plurality of pixel block pairs, each pixel blockpair including two pixel blocks disposed in point-symmetrical positionson opposite sides of the interpolated pixel, each of the pixel blocksincluding an identical number of said pixels, the pattern similarityvalue of each pixel block pair being calculated from values of thepixels in its two constituent pixel blocks; deciding whether similaredges are present in corresponding positions in the two pixel blocksconstituting each said pixel block pair, on the basis of differencesbetween the pixel values of pixels disposed on mutually adjacent scanlines in one of the pixel blocks, aligned in a direction perpendicularto the scan lines, and differences between the values of pixels disposedin corresponding positions in the other pixel block in the same pixelblock pair; selecting an interpolation direction corresponding to thepositions of the two pixel blocks in a pixel block pair having a patternsimilarity value indicating greatest similarity, among the pixel blockpairs in which similar edges are present; and generating theinterpolated pixel by using, as reference pixels, the pixels disposedclosest to the centers of the two pixel blocks in the pixel block paircorresponding to the selected interpolation direction.
 16. The method ofclaim 15, wherein generating the interpolated pixel further comprises:if there are pixels at the centers of the two pixel blocks in the pixelblock pair corresponding to the selected interpolation direction, usingthe two pixels disposed at the centers as reference pixels; and if nopixels are disposed at the centers of said two pixel blocks, using aplurality of pixels in each of the two pixel blocks as reference pixels.